Rebuild isolated Level 0 as realistic BPE nanoGPT baseline - #114

What changed

Rebuilds only the self-contained level_0_baseline/ subtree as a realistic MacBook-scale language-model baseline.

• replaces the obsolete 82K-parameter, one-block raw-byte experiment with GPT-2 BPE tokenization and a four-block, four-head, width-256 nanoGPT model;
• pins FineWeb-Edu and prepares immutable uint16 train, validation, and test token splits under /tmp;
• uses standard AdamW decay/no-decay parameter groups, 200-step linear warmup, cosine decay, weight decay, gradient clipping, and resumable checkpoints;
• separates training and evaluation random-number streams and uses fixed train/validation probes;
• evaluates untouched test data only at the final and validation-selected checkpoints;
• records deterministic, non-randomized WeightWatcher alpha diagnostics for transformer matrices;
• writes transactional completion metadata, final/selected test metrics, persistent logs, and plotting notebooks;
• retains an optional Muon path but makes AdamW the default isolated baseline.

Root cause

The previous isolated baseline simplified the scientific experiment rather than merely isolating the software path: it used raw bytes, one block, width 64, only about 82K parameters, repeatedly evaluated test data, and shared one RNG stream between evaluation and future training batches. The resulting ~26% next-byte accuracy plateau was a toy byte-model result, not a realistic nanoGPT baseline.

Default Level 0 protocol

• GPT-2 BPE, model vocabulary 50,304;
• four layers, four heads, width 256, context 256;
• approximately 16.1M trainable parameters;
• batch size 8 × four gradient-accumulation steps;
• 5,000 optimizer steps and 40.96M processed training tokens;
• 20M/1M/1M fixed FineWeb-Edu train/validation/test token splits;
• AdamW peak LR 6e-4, minimum LR 6e-5, weight decay 0.1, betas (0.9, 0.95), gradient clipping 1.0;
• WeightWatcher every 500 steps, with test evaluation only after training.

Scope

No existing WW-PGD, Level 0–2 framework, or other repository code is changed. All scientific changes are confined to level_0_baseline/. Temporary transfer files will be removed before merge.

Validation performed locally

• PYTHONPATH=src pytest -q — 6 passed;
• Python compilation passed;
• all shell scripts passed bash -n;
• both notebooks parsed as valid nbformat JSON;
• a bounded CPU end-to-end run wrote metrics, resumable checkpoints, final and validation-selected test metrics, and a completion marker.

The full 5,000-step streamed FineWeb-Edu run was not executed in the sandbox because external dataset download is unavailable there.